Surface computing systems enable users to interact with software applications by physical actions, rather than by issuing commands through a point-and-click menu hierarchy, or by speaking or typing commands. However, recognizing physical actions and correlating them correctly to commands intended by the users remains an ongoing challenge. Another challenge is reducing the impact of jitter inadvertently introduced into the physical actions by users.